Ablation Studies Submitted

13 SLURM jobs have been submitted to run comprehensive ablation studies for this NFM submission:

2×2 Matrix: NFM × Legal TTT

Isolating the individual contributions of NFM and legal TTT at matched 7k steps.

NFM Hyperparameter Sweeps

Loss weight sweep (hidden_dim=256, seed=42):

• lw=0.01 → jobs 55398696→55398699
• lw=0.05 → jobs 55398697→55398700
• lw=0.10 (default) → 1.12312 ✅
• lw=0.20 → jobs 55398698→55398701

Hidden dim sweep (loss_weight=0.1, seed=42):

• hd=128 → jobs 55398702→55398704
• hd=256 (default) → 1.12312 ✅
• hd=512 → jobs 55398703→55398705

Also pending

• 3-seed reproducibility runs (seeds 1337, 2025): jobs 55398556–55398561
• E2E TTT+Flow 7k reeval with 5h wallclock: job 55398555

Results will be updated in README as jobs complete.

Community Review — Non-record: 11L NativeFlowMatcher + Legal TTT — val_bpb 1.1199 (3-seed mean no-TTT: 1.1225)

BPB: 1.1199 | Compliance: FLAG — hashed n-gram cache with target-in-key (PR #779 family pattern)

What I found in the code (head SHA 039f4e891256, file records/track_non_record_16mb/2026-03-31_11L_NativeFlowMatcher_LegalTTT/train_gpt.py):

The n-gram lookup key at line 1475 is constructed by XOR-ing the target token into the hash:

line 1475: full_key = <hash> ^ (tgt_np * ng_primes[...]) & mask

This matches the full_key = ((ctx_hash ^ (target * primes[k])) & mask) construction that @valerio-oai ruled disallowed on PR #779 (comment 4145781641, 2026-03-27). Per the mechanism explanation, hashing the target token into the lookup key only reweights the correct token — in the hash-collision limit this drives P(correct) → 1 regardless of the data, which inflates the reported BPB without producing real compression.

Per Issue #1017 condition 1, p_t may depend only on the artifact and x_1...x_{t-1}. Because the lookup key at line 1475 is a function of the target token, the count read at scoring position t depends on x_t itself — which is the core violation the #779 ruling targets.

Cluster context: this same structural pattern has been closed on 15+ PRs under the #779 ruling as of 2026-04-11 (#779 itself, #770, #798, #808, #825, #786, #797, #909, #940, #761, #776, #788, #774, #778, #715, #758, #702 upstream, #1488). The base neural model is unaffected by this flag — in every case where the authors resubmitted without the n-gram cache, the base val_bpb has been in the ~1.10-1.15 range (standard for the SP1024 11L class).

CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.06s, dim=512, layers=11, vocab=1024, code=115032 B, SMOKE_TEST_PASS

Verdict: COMPLIANCE FLAG — target-in-key hashed n-gram cache, same family as PR #779.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: CLOSE under the same ruling as the rest of the family-bug cluster. A context-only resubmission (drop the target from the lookup key and use a full-vocabulary reweighting from a single context row, per @valerio-oai's suggested legal path on #779) would be welcomed.

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.06s, dim=512, layers=11, vocab=1024, code=115032 B, SMOKE_TEST_PASS. Classification via deterministic AST-based classify_prs.py (pattern bank derived from ~65 manually-reviewed PRs earlier in the 2026-04-11 sweep). This review was auto-drafted from a template and spot-checked before posting — if the template misread your code, please call it out so I can iterate the classifier.